fix(aws_s3 sink): close idle connections for aws s3 sinks #9703
Conversation
Fixes #9671 Signed-off-by: Vladimir Zhuk <[email protected]>
|
✔️ Deploy Preview for vector-project canceled. 🔨 Explore the source changes: ee46a62 🔍 Inspect the deploy log: https://app.netlify.com/sites/vector-project/deploys/616fdc979982af00082b9e4b |
vladimir-dd
changed the title
src/sinks/s3_common/config.rs
Outdated
| proxy, | ||
| // S3 closes idle connections after 20 seconds, | ||
| // so we can close idle connections ahead of time to prevent re-using them | ||
| client::Client::builder().pool_idle_timeout(Duration::from_secs(10)), |
There was a problem hiding this comment.
Small quibble: I'd change this to 15 seconds just to utilize a little more of the 20 second window.
There was a problem hiding this comment.
Can we not couple both timeouts with a shared constant or something? And shouldn't we explicitly set the 20 second timeout as well in case that changes in the future?
There was a problem hiding this comment.
I've not found any shared aws constant that we could rely on(I could create our own, but that wouldn't help much in case it changes on the aws side), so will just change it to 15sec.
src/sinks/s3_common/config.rs
Outdated
| proxy, | ||
| // S3 closes idle connections after 20 seconds, | ||
| // so we can close idle connections ahead of time to prevent re-using them | ||
| client::Client::builder().pool_idle_timeout(Duration::from_secs(10)), |
Signed-off-by: Vladimir Zhuk <[email protected]>
Fixes #9671 (potentially)
We suspect that S3 closes idle connections after 20sec while hyper preparing a new request. Hyper closes idle connections after 90sec by default, so we should decrease this timeout to something less than 20sec to avoid reusing closed connections. See this thread.
Signed-off-by: Vladimir Zhuk [email protected]